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Selected  Research  Opportunities  (SRO)  II 
Non-Gaussian  Workshop,  7  and  8  April  1981 
U.S.  Naval  Underwater  Systems  Center 
New  London  Laboratory,  New  London,  CT 


Introduction 


by  D.  M.  Viccione 

Naval  Underwater  Systems  Center 

New  London,  CT 


Conventional  acoustic  and  electromagnetic  antisubmarine  warfare  (ASW) 
systems  are  designed  to  optimally  detect  the  presence  of  signals  in  Gaussian 
background  noise.  However,  in  the  electromagnetic  environment  (EM)  high  level 
directional  transient  or  other  structured  noise  is  a  familiar  problem.  Such  in¬ 
terferences  are  typically  non-Gaussian.  Frequently  receiver  systems  designed  for 
Gaussian  noise  backgrounds  are  seriously  degraded  in  the  non-Gaussian  en¬ 
vironment.  During  the  past  5  years,  considerable  attention  has  been  focused  on  the 
(1)  problem  of  characterizing  non-Gaussian  noise  fields  and  (2)  design  of  EM 
detectors  (receivers)  that  operate  optimally  in  such  environments. 

The  necessity  of  operating  acoustic  surveillance  and  communication  systems  in 
typical  non-Gaussian  noise  backgrounds  must  be  anticipated.  Such  noise  may  arise 
naturally  from,  for  example,  oil  exploration  and  drilling  in  the  deep  sea,  or 
deliberately  from  intentional  jamming,  such  as  detonations  of  high-yield  explosives 
in  ocean  basins. 

The  Office  of  Naval  Research  (ONR)  has  initiated  a  new  feature,  the  Selected 
Research  Opportunities  (SRO)  Program,  in  its  Contract  Research  Program  (CRP). 
The  SRO  Program,  like  the  CRP,  is  designed  to  further  the  goal  of  improved 
national  defense  over  the  long  term.  This  is  to  be  accomplished  through  (1)  in¬ 
creased  involvement  of  the  academic  research  community  in  selected  fundamental 
research  areas  and  (2)  fostering  stronger  links  between  this  community  and  the 
Navy.  One  of  the  research  areas  in  the  SRO  Program  is  that  of  non-Gaussian  signal 
processing,  which  offers  an  opportunity  to  explore  the  feasibility  and  practicality  of 
designing  and  developing  ASW  systems  to  operate  efficiently  in  non-Gaussian 
noise.  Generically  the  issues  to  be  resolved  by  the  non-Gaussian  research  give  rise  to 
the  following  objectives: 

1 .  To  analyze  existing  algorithms  for  worst  case  situations  in  the  presence  of 
intentional  man-made  noise.  Nonparametric  and  robust  approaches  should  be 
explored  to  devise  algorithms  that  perform  well  regardless  of  noise  source 
characteristics. 
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2.  To  investigate  techniques  for  weak  signal  extraction  in  the  presence  of  strong 
signals;  and,  as  a  special  case,  explore  techniques  for  detecting  spread-spectrum 
signals,  including  analysis  of  changes  in  noise  statistics  caused  by  pseudo-noise 
signals. 

3.  To  enlarge  the  basic  mathematical  and  statistical  theory  of  non-Gaussian 
stochastic  processes  applicable  to  EM  and  underwater  sound  environments. 

4.  To  study  non-Gaussian  models  appropriate  for  reverberation  or  rever- 
beration-like  noise  in  shallow  water. 

5.  To  examine  the  extent  of  non-Gaussian  sources  of  impulsive  noise  in¬ 
terference  in  underwater  acoustic  signal  processing  problems  of  interest  to  the 
Navy. 

6.  To  overcome  shortcomings  in  existing  theories  concerning  nonwhite  in¬ 
terference,  nonstationarity  of  channels  and  interferences,  multipath,  dispersion, 
Doppler  effects,  and  problems  caused  by  sensor  and  array  flaws. 

The  workshop,  jointly  sponsored  by  ONR  and  the  Naval  Underwater  Systems 
Center  (NUSC),  was  the  first  to  bring  together  SRO  II  participants  and  others  in  the 
processing  community  who  are  working  in  the  area  of  non-Gaussian  noise.  Sum¬ 
maries  of  each  technical  presentation  are  provided  in  this  document  and  a  list  of 
participants  is  included  to  facilitate  communication.  The  schedule  for  the 
presentation  of  the  papers  at  the  SRO-II  Workshop,  held  on  7-8  April  1981  at 
NUSC  New  London,  CT,  follows. 
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The  Performance  and  Robustness  of  Suboptimum  and 
Locally  Optimum  Detectors  in  Non-Gaussian  Noise 


by  A .  D.  Spaulding 

U.S  Dept,  of  Commerce  (NTIA) 

Boulder,  CO 


Most  currently  used  receiving  systems  are  those  that  are  optimum  in  Gaussian 
noise.  Unfortunately,  the  actual  interference  environment  is  almost  never  Gaussian 
in  character,  but  usually  quite  different,  being  impulsive  in  nature.  By  impulsive,  we 
mean  that  there  are  small  probabilities  of  quite  large  instantaneous  values  of  noise. 
This  is  a  general  definition  in  that  there  exist  both  broadband  (the  usual  definition) 
and  narrowband  impulsive  processes.  When  confronted  with  this  real-world  in¬ 
terference,  the  usual  procedures  were  to  attempt  to  change  the  interference,  as  seen 
by  the  receiver,  to  look  more  Gaussian  in  character.  Such  attempts  usually  em¬ 
ployed  various  ad  hoc  nonlinearities  preceding  the  detector.  While  these  are  oc¬ 
casionally  reasonably  effective,  they  can  be  extremely  wasteful  of  spectrum  space 
when  normal  signal-to-noise  ratios  are  considered.  As  we  will  see,  these  ad  hoc 
nonlinearities  can,  under  certain  conditions,  be  almost  as  good  as  the  optimally 
derived  nonlinearity  for  the  limiting  case  of  a  vanishing  small  signal.  (It  is  believed 
that  this  result  may,  however,  be  misleading  and  it  requires  further  investigation.) 

Only  recently  have  there  been  receiving  systems  designed  to  match  the  actual 
interference.  These  truly  optimum  systems  are  usually  difficult  to  realize  physically. 
It  is  possible,  however,  if  the  desired  signal  is  small,  to  obtain  realizable  systems 
that  approach  true  optimality.  These  are  generally  termed  locally  optimum.  The 
receiving  systems  must  be  adaptive,  adjusting  themselves  to  changing  interference 
conditions.  That  is,  they  must  estimate  the  appropriate  parameters  for  the  noise 
models  from  which  the  system  was  designed.  The  design  is  usually  done  by 
techniques  termed  locally  optimum  Bayes  detection  (LOBD).  The  parameter 
estimation  problem  is  currently  being  investigated  via  locally  optimum  Bayes 
estimation  (LOBE)  (Middleton  (to  be  published)]. 

It  is  the  purpose  of  this  presentation  to  investigate  the  robustness  of  these  locally 
optimum  systems.  That  is,  we  want  to  determine  the  degree  of  accuracy  required  in 
the  parameter  estimation  procedure.  For  illustration  we  will  use,  initially,  two 
examples  of  interference,  one  broadband  and  one  narrowband,  and  consider  the 
simplest  case  of  coherent  signal  detection.  The  standard  LOBD  analysis  is  reviewed 
and  then  used  to  determine  the  performance  for  arbitrary  interference  with  the 
detector  that  is  locally  optimum  for  any  given  interference  (our  estimate).  Of 
course,  if  the  actual  interference  and  the  estimate  are  identical,  we  obtain  the 
standard  results.  Next,  we  study  in  some  detail  the  two  most  common  ad  hoc 
nonlinearities  (the  hard-limiter  and  the  adaptive  clipper)  to  determine  their  small 
signal  performance  via  the  LOBD  results.  Then,  we  present  a  series  of  sample 
calculations  comparing  the  performances  of  the  hard-limiter  and  LOBD 


nonlinearities  and  the  robustness  (for  the  narrowband  noise  example)  of  the  LOBD 
nonlinearity.  It  is  suggested  that  the  small  signal  limiting  results,  in  some  cases  at 
least,  may  be  misleading.  We  show  this  by  presenting  some  system  performance 
results  that  seem  to  be  contrary  to  the  limiting  results  (which  are  obviously  true) 
when  small  (but  nonzero)  signal  levels  are  considered. 

The  LOBD  analysis  employed  to  obtain  the  robustness  results  and  the  com¬ 
parison  of  performance  between  LOBD’s  and  various  nonlinear  ad  hoc  detectors 
requires  use  of  the  central  limit  theorem  and  are  strictly  valid  only  as  the  signal 
becomes  vanishingly  small.  In  special  cases,  the  broadband  interference  model 
utilized  (Middleton’s  Class  B  Noise  Model)  reduces  (approximately)  to  the 
mathematically  simpler  Hall  Noise  Model.  The  analysis  is  repeated  for  the  Hall 
model  and  performance  estimates  are  obtained  for  the  LOBD  and  hard-limiter 
without  resorting  to  the  vanishingly  small  signal  assumption.  It  is  shown  that 
performance  results  obtained  via  the  vanishingly  small  signal  assumption  are  valid, 
at  least  for  the  special  cases  considered,  as  long  as  the  time-bandwidth  product  is 
reasonably  large  (>  30). 


Non-Gaussian  Interference  Environments, 
State-of  the* Art  Remarks 

by  D.  Middleton 
128  East  9 1st  St. 

NY,  NY 


An  overview  of  recent  work  by  the  author  on  non-Gaussian  noise  models  and 
threshold  signal  processing  involving  non-Gaussian  electromagnetic  interference 
(EMI)  is  presented  here.  The  principal  purpose  is  to  offer  a  technical  update  of 
results  obtained  recently,  with  some  of  their  implications  for  a  quantitative 
description  of  the  EMI  environment  pertinent  to  Navy  uses.  Both  acoustic  and  EM 
applications  would  be  included.  Although  a  complete  overview  is  not  made  or 
intended,  various  specialized  interests  should  find  appropriate  topics  of  interest 
without  difficulty. 

The  current  material  is  compiled  from  a  series  of  recent  presentations  by  the 
author  (1980, 1981)  in  the  form  of  vugraphs.  The  main  topics  are 

1 .  construction  of  analytically  tractable  non-Gaussian  noise  models, 

2.  optimum  threshold  detection  algorithms  for  non-Gaussian  EMI  en¬ 
vironments,  and 

3.  evaluation  of  optimum  and  suboptimum  receiver  performance. 

The  suboptimum  receivers  considered  are  optimum  in  Gaussian  noise  (e.g., 
correlation  detectors  that  are  seen  to  be  very  much  degraded  in  highly  non-Gaussian 
environments).  Finally,  the  critical  role  of  the  EMI  scenario  is  discussed,  whereby 
the  parameters  of  the  noise  model  may  be  constructed  from  statistical-physical 
consideration  and  measured  empirically. 
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Performance  of  the  Optimum  and  Several  Suboptimum 
Receivers  for  Threshold  Detection  of  Known  Signals 
in  Additive,  White,  Non-Gaussian  Noise 

by  R.  F.  Ingram 

Naval  Underwater  Systems  Center 
New  London,  CT 


The  additive  noise  encountered  at  a  receiver  input  is  often  non-Gaussian.  If  the 
non-Guassian  nature  of  the  noise  is  taken  into  account  in  the  design  of  the  receiver, 
significant  performance  improvements  relative  to  the  performance  of  the  linear 
receiver  can  be  achieved.  This  presentation  discusses  the  performance  im¬ 
provements  that  can  be  expected  from  the  optimal  and  several  subpotimal  receivers 
frequently  utilized  for  detecting  known  threshold  signals  in  an  additive,  white,  non- 
Gaussian  noise. 

One  form  of  the  optimum  receiver  for  detecting  known  threshold  receivers  in 
additive,  white,  non-Gaussian  noise  consists  of  the  receiver  that  should  be  utilized  if 
the  noise  were  Gaussian,  except  that  a  nonlinearity  is  placed  between  the  receiver 
input  and  the  Gaussian  detector.  The  input-output  characteristic  of  the  nonlinearity 
is  given  by  -d/dx(ln  p(x)),  where  p(x)  is  the  1st  order  probability  density  function  of 
the  noise  alone.  The  measure  of  performance  improvement  obtained  by  including 
the  nonlinear  device  is  given  by  the  ratio  of  the  signai-to-noise  ratio  (SNR)  at  the 
receiver  output  with  and  without  a  nonlinear  processor.  The  magnitude  of  this 
improvement  for  threshold  signals  is  evaluated  using  the  1st  order  probability 
density  functions  resulting  from  Middleton’s  Class  B  Noise  Model.  The  per¬ 
formance  of  several  suboptimal  but  more  easily  implemented  nonlinearities,  i.e. 
clipper,  hole  puncher,  and  hard  limiter,  are  also  evaluated  and  compared  to  the 
optimal’s  performance.  Finally  the  performance  of  these  threshold  receivers  are 
evaluated  as  a  function  of  the  receiver  input  SNR. 
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Frequency  Tracking  and  Parametric  Spectrum  Analysis 


by  L.  Scharf 

Colorado  State  University 
Department  of  Electrical  Engineering 
Ft.  Collins,  CO 


In  this  presentation  we  review  several  key  ideas  in  random  phase  tracking, 
parametric  spectrum  analysis,  and  prediction  in  infinite  variance  random  processes. 
It  is  shown  that  random  phase-  and  frequency-modulated  data  may  be  tracked  with 
a  dynamic  programming  algorithm  that  recursively  maximizes  Bayesian  recursions 
to  generate  surviving  sequences  that  ultimately  can  qualify  as  maximum  likelihood 
sequences.  The  discussion  of  parametric  spectrum  analysis  focuses  on  a 
parameterized  version  of  the  Bartlett  and  maximum  likelihood  spectra,  and  the 
performance  that  can  be  expected.  The  discussion  of  prediction  in  infinite  variance 
processes  deals  with  procedures  for  constructing  predictors  and  whiteners  when  no 
finite  prediction  error  variance  is  available  for  minimization. 
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Non-Gaussianity  of  Oceanic  Internal  Waves 

by  D.  McClovic 
Arrete Assoc 
P.O.  Box  350 
Encino,  CA 

Introduction 


The  background  noise  field  of  concern  is  the  ambient  field  of  internal  waves  and 
sporadic  turbulence.  The  horizontal  scales  of  these  phenomena  range  from  tens  of 
kilometers  to  less  than  the  present  instrumentation  limits  (centimeters).  The  natural 
fluctuations  in  the  environment  can  be  conveniently  characterized  in  the  form  of  1- 
D  and  2-D  displacement  spectra. 

One-dimensional  horizontal  displacement  spectra  obtained  at  different  times  and 
places  in  the  seasonal  thermocline  consistently  have  slopes  between  -2  and  -2.5,  with 
the  latter  being  more  common  on  the  scale  of  4-10  m.  When  the  spectral  estimates 
are  averaged  over  a  large  (MO  km)  horizontal  extent  in  a  given  locale,  they  exhibit 
apparent  stability  and  approximate  universality.  However,  the  spectral  levels  can 
vary  considerably  over  horizontal  scales  of  500  m,  even  when  the  data  from  many 
sensors  are  averaged.  Two-dimensionsl  displacement  spectra  have  been  estimated 
from  thermistor  chain  tows  and  the  spectral  shapes  are  in  agreement  with  the 
McClovic  model,  i.e., 
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Examples  of  the  spectra  are  given  in  the  following  figures. 
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Estimate  internal  wave  displacement  from  temperature 
fluctuations  T^(x): 


di(x) 


^(x)  -  <Ti> 
<dT/dz>.. 


<dT/dz>^ 


z  •  -  z  • 

i+m  i-m 


Figure  1.  Oceanic  Internal  Waves 
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Figure  2.  Temperature  Displacement  Raster 
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Figure  3.  Average  Horizontal  Power  Spectrum  of  Temperature  Displacement 
Data  (Horizontal  Data  Patch  of  5.26  km;  Sample  Interval  of  1 .28  m) 
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(300-1000)  PREDICTED  GAUSSIAN  LEVEL  AT  k=0 
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Figure  5.  Spectral  Density  for  the  Normalized,  Whitened,  Bandpassed  Power 
Average  Over  Depth  (100-1000  m  Band) 
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Figure  6.  Spectral  Density  for  the  Normalized,  Whitened,  Bandpassed  Power 
Averaged  Over  Depth  (4-100  m  Band) 


TD  6591 


Approach 


A  model  has  been  formulated  to  account  for  the  large  fluctuation  observed  in  the 
displacement  data.  In  particular,  a  nonlinear  temperature  profile  is  used  to  account 
for  the  degree  of  non-Gaussianity  observed  in  the  temperature  fluctuations, 
presuming  that  the  internal  wave  field  is  Gaussian. 

The  temperature  model  is  described  as  follows  (see  figure  7): 


T(x,z)  =  Tq(x,z  -  C(x,z)), 
with  Tq  and  5  Gaussian  processes 


TCMTCRATUM 


Figure  7.  Temperature  Model 
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T(x,z)  ■  T0(x»z  ”  5 (x, z) ) 


Look  at  the  whitened  temperature: 


3T 

3x 


3T, 


3x 


3x 


Define  deviation  from  linearity 


3T  3T 

3Z2  =  <TT>  (1  +  9)'  30  that 


d'  (x) 


-1  3Tq 

53T 


+ 


a  +  e)c* 


High  passd':  d'-*ds' 


a;  *  <i  +  +  'V’s 


where 


9  '  8L  +  V 


Look  at  the  effect  of  long  scales  in  6  on  the  non-Gaussianity  of  short  scale  fluc¬ 
tuations  d ' 


dg (x)  *  ti  +  eL(*m;(x) 


assuming  0L£s'  Gaussian  with  PSD  P9(k),  P4(k). 


16 


the  statistics  of  displacement  power 
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/u  individual  cases  covering  40  m  in  depth.  The  horizontal  extent 
of  1  he  data  is  5.26  km;  the  sample  interval  is  1 .28  m.  Predictions 
of  the  power  spectral  levels  from  the  microstructure  model  are 
included  for  two  values  of  ke  with  <62>  =  1.) 
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Acoustic  Problems 


by  R.  F.  Dwyer 

Nava /  Underwater  Systems  Center 
New  London,  CT 


This  presentation  describes  the  results  of  a  statistical  analysis  study  of  FRAM  II 
Arctic  under  ice  ambient-noise  data.  The  specific  data  analyzed  were  recorded  on 
23-24  April  1980  from  a  pack-ice  camp  in  the  Arctic  Ocean  at  86°  N  latitude,  25  °  W 
longitude.  At  this  location,  the  bottom  depth  was  approximately  4000  m.  The 
measurement  system  consisted  of  a  broadband  omnidirectional  hydrophone 
suspended  to  a  depth  of  91m  from  a  sonobuoy  located  in  a  lead.  Under  the  influence 
of  Arctic  currents,  the  pack  ice  was  slowly  moving.  This  movement  caused  rifting 
and  cracking  of  ice  that  occurred,  at  times,  throughtout  the  experiments  and 
represented  a  structured  acoustic  noise  source.  Both  impulsive  and  burst  noises  were 
identified  in  the  data  and  were  probably  caused  by  tensile  cracks  and  rubbing  ice 
masses. 

To  better  understand  the  statistical  properties  of  under  ice  ambient  noise,  the 
skew,  kurtosis,  and  cumulative  distribution  function  (CDF)  of  the  data  were 
estimated.  In  the  time  domain,  the  statistics  were  estimated  in  100-,  350-,  and  2500- 
Hz  bands.  At  times,  the  statistical  estimates  in  all  bands  deviated  from  Gaussian 
noise  significantly,  and  were  consistent  with  previously  reported  results  of  ex¬ 
periments  made  in  the  Canadian  Arctic  Archipelago.  The  estimated  energy  CDF  of 
FRAM  II  data  predicted  detection  thresholds  3  to  10  dB  higher  than  what  would  be 
expected  from  purely  Gaussian  phenomena.  Spectrum  levels  and  spectrograms  were 
also  measured.  The  spectrograms  depicted  dynamic  frequency  components  that 
appear,  from  aural  information  that  sounded  like  squeaks,  to  be  correlated  with  ice 
dynamics.  Comparisons  of  broadband  spectrum  level  estimates  at  different  times, 
indicate  nonstationary  frequency  domain  components  that  also  appear  to  be 
correlated  with  ice  dynamics. 

Since  it  was  known  that  burst-noise  durations  of  Arctic  under  ice  noise  last  from 
0.1  to  1/3  s,  statistical  estimates  of  frequency  domain  components  were  measured. 
These  frequency  domain  statistical  measurements  represent  new  techniques  for 
estimating  environmental  noise  phenomena.  The  complex  skew,  kurtosis,  and  CDF 
were  measured  in  1-,  2-,  6-,  and  10-Hz  resolution  cells  at  the  output  of  a  discrete 
Fourier  transform,  employing  processing  times  of  2  to  14  min.  These  new  findings 
indicate  the  existence  of  strong  non-Gaussian  noise  in  the  frequency  domain. 
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Data- Adaptive  Principal  Component  Signal  Processing 


D.  W.  Tufts 

Department  of  Electrical  Engineering 
University  of  Rhode  Island 
Kingston,  RI 


Even  the  best  linear-prediction-based  methods  for  fitting  multiple-sinusoid  signal 
models  to  observed  data,  such  as  the  forward-backward  methods  of  Nuttall 
(reference  1)  and  Ulrych  and  Clayton  (reference  2),  are  ill-conditioned.  The 
locations  of  estimated  spectral  peaks  can  be  greatly  affected  by  additive  noise.  This 
ill-conditioning  can  be  alleviated  by  singular  value  decomposition  of  the  linear- 
prediction-data  matrix.  (This  is  a  preliminary  version  of  a  presentation  to  be  made 
at  the  First  ASSP  Workshop  on  Spectral  Estimation,  17-18  August  1981,  McMaster 
University,  Hamilton,  Ontario.) 

References 

1.  A.H.  Nuttall,  Spectral  Analysis  of  a  Univariate  Process  With  Bad  Data 
Points  via  Maximum  Entropy  and  Linear  Predictive  Techniques  in  NUSC  Scientific 
and  Engineering  Studies,  Spectral  Estimation,  Naval  Underwater  Systems  Center, 
New  London,  CT,  March  1976. 

2.  T.  J.  Ulrych  and  R.W.  Clayton,  Time-Series  Modeling  and  Maximum 
Entropy,  Physics  of  the  Earth  and  Planetary  Interiors,  vol.  12,  August  1976,  pp. 
188-200. 

Bibliography 

1.  Tufts,  D.  W.,  and  Kumaresan,  R.,  Improved  Spectral  Resolution  II, 
Proceedings  of  the  IEEE  International  Conference  on  Acoustics,  Speech,  and 
Signal  Processing,  April  1980,  pp.  592-597. 


21 


Residual  Signal  Analysis-A  Search  and  Destroy 
Approach  to  Spectral  Analysis 


J.P.  Costas 
General  Electric  Co. 

Court  St.  Plant,  Syracuse,  NY 


A  signal  processing  procedure  especially  suited  for  use  in  narrowband  spectral 
analysis  or  line  tracking  applications  is  presented.  A  modular  approach  is  employed 
that  permits  each  tracker  to  optimize  estimation  parameters  for  the  line  assigned. 
The  signal  estimate  from  each  module  is  used  to  cancel  the  corresponding  signal 
from  the  common  bus  to  which  all  tracker  inputs  are  connected.  A  feedback 
arrangement  around  each  module  restores  to  each  tracker  the  full  level  of  its 
assigned  input  signal.  Thus,  each  signal  being  tracked  is  prevented  from  causing 
interference  to  any  other  tracker.  The  advantages  obtained  from  this  arrangement 
are  demonstrated  and  discussed. 


Optimal  Identification  of  Nonlinearly 
Transformed  Gaussian  Processes 


by  C.L.  Hindman 

TR  W  Defense  and  Space  Systems  Group 
One  Space  Park,  Los  Angeles,  CA 


In  a  variety  of  applications  it  is  necessary  to  deal  with  observations  of  a  time 
series  obtained  via  an  instantaneous  nonlinear  mapping  of  an  underlying  random 
variable  whose  values  cannot  otherwise  be  ascertained.  In  the  case  where  the 
nonlinear  mapping  is  known  and  can  be  unambiguously  inverted,  such  a  com¬ 
plication  represents  little  difficulty.  However,  in  some  applications,  a  priori 
knowledge  of  the  intervening  nonlinear  mapping  is  either  incomplete  or  altogether 
lacking.  In  such  cases  any  analysis  of  the  underlying  random  process  must  be  ac¬ 
companied  by  the  identification  or,  more  correctly,  the  estimation  of  one  or  more 
features  of  the  nonlinear  map  or  measurement  function  itself. 

This  presentation  is  concerned  with  the  practicalities  of  this  identification 
problem  in  the  case  where  the  underlying  random  process  is  Gaussian  or,  rather, 
Gauss-Markov  in  nature  and  generally  exhibits  strong  serial  or  time  correlation.  It  is 
this  factor  of  time  correlation  that  renders  the  procedures  discussed  herein  of  in¬ 
terest,  for  in  the  case  of  independent  observations  the  determination  of  the 
nonlinear  map  is  essentially  equivalent  to  the  estimation  of  the  univariate  density 
junction  of  the  observations  themselves.  In  the  serially  correlated  case,  however,  the 
most  efficient  use  of  the  observations  will  involve  simultaneous  estimation  of  the 
nonlinear  function,  as  well  as  the  structure  of  the  correlation  of  the  underlying 
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process.  In  this  sense  the  procedure  advocated  here  may  be  regarded  as  a  general 
scheme  for  fitting  nonlinear  time  series  models,  albeit  the  nonlinearity  is  of  a  rather 
restricted  type.  A  few  practical  circumstances  in  which  the  procedure  discussed  may 
be  of  value  are  indicated  briefly  as  follows: 


1.  Measurement  by  a  sensor  of  unknown  characteristic.  When  the  response  of 
a  sensor  is  unknown,  but  one  has  available  sensor  measurements  of  an  underlying 
random  process  of  known  character,  then  we  are  confronted  with  the  problem  of 
identifying  both  the  sensor  and  estimating  the  underlying  processes  simultaneously; 
i.e.,  if  y  is  the  sensor  output  we  have 


y(t)  *  h(x(t)), 


(1.1) 


where  h(x)  is  the  sensor  response  to  be  determined  from  the  measurements  y(t). 

2.  Indirect  determination  of  internal  motion  via  measurement  of  an  imbedded 
scalar  in  a  stratified  medium.To  trace  the  dynamic  development  of  the  local  fluid 
vertical  velocity  in  the  thermally  stratified  atmosphere  by  means  of  a  time  series 
temperature  measurement  at  fixed  altitude,  we  hypothesize  the  measurement 
relation 

T(t)  =  T0{Z  (1.2) 

where  T(t)  is  the  recorded  time  series  of  temperature,  Z  is  the  sensor  altitude,  w(t)  is 
the  instantaneous  vertical  fluid  velocity  at  measurement  altitude  Z,  and  T(h)  is  the 
equilibrium  or  ambient  thermal  lapse  or  profile  as  a  function  of  altitude,  which  is  to 
be  regarded  as  unknown  except  through  the  measurements  T(t). 

3.  Measurement  of  concentration  fluctuations  of  reactive  species.  When  two 
or  more  chemically  reactive  species  are  randomly  distributed  in  a  medium,  the 
equilibium  concentration  of  species  A,Ca  and  another  species  B,Cb  may  be  related 
via  an  equilibrium  equation  of  the  form 

cA  -  f(cB,e), 

where  0  is  a  set  of  unknown  or  incompletely  known  parameters.  When,  for  instance, 
knowledge  of  CB  is  desired,  but  CA  is  more  conveniently  measurable,  then  we  must 
invert  the  nonlinear  relationship  to  obtain  the  CB  fluctuation  history. 

4.  Nonlinear  time  series  modeling.  In  some  instances  it  is  useful  to  simply 
assume  the  existence  of  an  instantaneous  functional  relationship  between  an  ob¬ 
served  series  and  an  underlying  hypothetical  series  even  when  no  justifiable  basis  for 
such  a  transformation  can  be  advanced.  Such  ideas  are  proposed  frequently  in  the 
analysis  of  economic  time  series  when  it  is  desired  to  transform  a  given  series  into 

one  that  possesses  more  desirable  characteristics  from  the  analyst’s  point  of  view. 

• 

The  common  underlying  feature  of  these  examples  is  the  necessity  of  determining 
the  structure  of  the  nonlinear  measurement  relation.  This  is  done,  more  or  less,  on 
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the  basis  of  the  observations  alone  in  the  absence  of  a  priori  detailed  information 
concerning  the  statistics  of  the  underlying  process  that  generated  the  measurements 
in  the  first  place  (e.g.,  the  spectrum  or  covariances).  Since  the  problem  of  deter¬ 
mining  both  the  nonlinear  transformation  and  the  underlying  realization  is  highly 
nonunique,  we  expect  that  useful  results  will  be  obtainable  only  when  the  form,  the 
general  structure,  or  the  probability  distribution  of  the  underlying  process  can  be 
regarded  as  given  at  least  approximately.  Such  will  be  the  assumption  made 
throughout  this  presentation;  however,  we  make  the  observation  that,  just  as  the 
maximum  likelihood  estimation  of  linear  Gaussion  time  series  model  parameters 
has  a  least  squares  interpretation  even  for  non-Gaussion  distributions,  we  expect 
similar  utility  and  robustness  in  the  present  context.  This  will  be  true  even  when  the 
underlying  distribution  is  not  known,  so  long  as  minimization  of  the  residual  sum  of 
squares  leads  to  adequate  estimation  of  the  covariance  structure. 

The  basic  assumptions  concerning  the  nonlinear  mapping  to  be  made  are  of 
concern,  and  here  we  are  guided  by  practice  and  intuition.  The  required  basic 
features  are  uniqueness  and  smoothness.  By  uniqueness  we  mean  that  the 
measurements  are  related  to  the  underlying  process  in  a  one  to  one  fashion  over  the 
range  of  actually  observed  values.  Of  course,  maps  of  a  more  complex  structure 
could  be  treated  if  additional  a  priori  information  were  available. 

Requisite  properties  of  smoothness  are  somewhat  more  difficult  to  define 
precisely.  Certainly  for  discrete  parameter  data  the  empirical  sampling  density  of 
the  observations  as  a  function  of  the  observed  value  must  be  assumed  to  be  such 
that  variations  of  the  inverse  map  g(y)  =  x  (y  =  h(x))  are  not  severely  aliased  if  the 
behavior  of  g(y)  is  to  be  adequately  discerned  from  the  data  on  hand.  For  the  actual 
computational  implementation  considered,  we  use  the  stronger  assumption  of 
continuous  piecewise  linearity;  although  in  some  applications  this  assumption  may 
be  inappropriate.  However,  it  will  emerge  that  this  assumption  provides  con¬ 
siderable  computational  advantage,  as  well  as  being  an  appealing  choice  when  the 
transformation  itself  is  known  or  suspected  to  be  comprised  of  random  (and 
possibly  independent)  positive  increments  in  y  and  x.  In  such  cases  smooth 
representation  such  as  polynominals,  power  series,  or  continuous  orthogonal 
functions  are  generally  found  to  be  less  adequate,  as  well  as  more  difficult  to 
manage  computationally. 

When  a  more  computationally  exhaustive  search  is  contemplated,  the  use  of 
piecewise  linear  approximation  enjoys  the  additional  advantage  of  being  a  suitable 
basis  for  uniform  approximation  of  continuous  functions  of  bounded  variation. 
Therefore,  it  appears  to  be  a  logical  choice  for  approximating  a  monotone  con¬ 
tinuous  function.  In  this  case,  the  form  assumed  by  the  relevant  probability  func¬ 
tional  will  also  be  seen  to  have  the  further  computational  advantage  of  enabling 
direct  hypothesis  testing  on  the  choice  of  intervals  employed  in  constructing  the 
piecewise  linear  approximation.  This  allows  at  least  the  potential  for  dealing  with 
what  is  usually  the  most  disadvantageous  aspect  of  piecewise  linear  approximation-- 
namely,  the  choice  of  node  points  of  a  discontinous  derivative. 

The  basic  procedure  advocated  in  this  presentation  is  the  application  of  the 
principle  of  maximum  likelihood  to  the  simultaneous  estimation  of  the  nonlinear 
map  and  the  statistics  (in  the  Gaussian  case)  of  the  underlying  process.  This 
estimator  will  be  seen  to  embody  a  criterion  of  optimality  that  arises  from  the 
theoretical  analysis  of  the  effect  of  instantaneous  nonlinear  transformation  in  the 
Gaussian  case. 
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